Tangible Mixed Reality Desktop for Digital Media Management

This paper presents a tangible mixed reality desktop that supports gesture-oriented interactions in 3D space. The system is based on computer vision techniques for hand and finger detection, without the need for attaching any devices to the user. The system consists of a pair of stereo cameras that point to a planar surface as the work bench. Using stereo triangulation, the 3D locations and directions of the user’s fingers are detected and tracked in the space on and above the surface. Based on our 3D finger tracking technique, we design a few simple multi-finger gestural interactions for digital media management. The system provides a convenient and user-friendly way of manipulating virtual objects in 3D space and supports seamless interactions with physical objects.